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Abstract — The maximum-marginal-a-posteWon success rate of 
statistical decision under multivariate Gaussian error distribution 
on an integer lattice is almost rigorously calculated by using 
union-bound approximation and Monte Carlo integration. These 
calculations are applied to the revelation of the various possible 
realizations of the reliable and short-period integer ambigu- 
ity resolution in precise carrier-phase relative positioning by 
GPS/GNSS. The theoretical foundation and efficient methodology 
are systematically developed, and two types of the enhancement 
of union-bound approximation are proposed and examined. 

The results revealed include an extremely high reliability 
under the condition of accurate carrier-phase measurements 
and a large number of visible satellites, its heavy degrada- 
tion caused by the slight amount of differentiated ionospheric 
delays due to the nonvanishing baseline length between rover 
and reference receivers, and the advantages of the use of the 
multiple carrier frequencies. The succeeding initialization of the 
integer ambiguities is shown to overcome the disadvantageous 
condition of the nonvanishing baseline length effectively due 
to the reasonably assumed temporal and spatial constancy of 
differentiated ionospheric delays. 

Index Terms — Bayes decision, lattice basis reduction, Fincke- 
Pohst algorithm, union bound, Monte Carlo integration, GPS, 
GNSS, carrier-phase positioning, integer ambiguity resolution, 
ionospheric delay. 

CORRECT resolution of integer ambiguities is indispens- 
able for precise carrier-phase relative positioning with the 
Global Positioning System (GPS) and the global navigation 
satellite system (GNSS) [l]-[3]. This means that one needs 
an extremely high success rate and an extremely low error 
rate of the resolution, even in the heavily difficult resolution 
for non-permanent, or mobile, receivers used in realistic and 
unstable environments [4]. 

The relative positioning requires rover and reference re- 
ceivers, and precisely measures the differentiated coordinates 
of the position, i.e., the baseline vector spanned between the 
positions of the two receivers. Consequently, the environ- 
mental conditions for the rover receiver are crucial for the 
ambiguity resolution in normal cases. 

In practice, the improvement in the ambiguity resolution is 
inductively known to be achieved by using accurate carrier- 
phase and code-pseudorange measurements, a large number of 
visible satellites, multiple carrier frequencies, and other means. 
Although such conditions improve the statistical success and 
error rates, their reliable estimation seems not to have been 
thoroughly investigated. This is partially due to the fact 
that the theoretical calculations are extremely laborious high- 



dimensional integrations of a multivariate error probability 
distribution function in a complicated integration domain. 

The theoretical problems of the ambiguity resolution are 
formalized as optimal statistical decisions, and these are 
classified as either fixed-sample decisions or sequential ones 
according to the treatment of time series of the measurements, 
i.e., statistical samples. The first kind of decision uses assigned 
fixed-number epochs of measurements and generally chooses 
between making and avoiding a judgment. The second kind 
of decision is allowed to re-sample the measurement data 
sequentially. If a judgment is made, its optimal decision should 
choose the maximum marginal a posteriori solution for both 
kinds of decisions according to the Bayes statistical theory 
[5]. It is identical to the least-square solution in the case of 
the simple linear and Gaussian error distribution model. Several 
studies in the field of GPS/GNSS have discussed the theoret- 
ically optimal fixed-sample resolutions of integer ambiguities 
and the calculations of the success rate [6]-[8]. Other studies 
have discussed the sequential resolutions and the calculations 
of the success rate [9]-[ll]. Additional studies have described 
the efficient computational algorithms to search the integer 
least-square solution [7], [12]-[14]. However, neither the rig- 
orous aspect of statistical optimality nor the depth calculation 
of success and error rates seems to have been thoroughly 
investigated as yet [15] in the field of GPS/GNSS. 

Although the sequential resolution is required for the real- 
time positioning of mobile receivers, this study concentrates 
on the fixed-sample resolution, because it is required for the 
off-line data processing, and, furthermore, is considered to be 
fundamental to the study of the sequential one. 

Various conditions which are intended to improve the 
performance of the integer ambiguity resolution have been 
discussed in several studies. Such conditions include the use 
of precise code-pseudorange and carrier-phase measurements 
[16], [17], the short baseline length of relative positioning 
[18], the use of a reference receiver network [19]-[21], the 
use of multiple carrier-phase measurements, or carrier frequen- 
cies [22], more visible satellites, including planned European 
Galileo navigation ones [23], [24], the preferable satellites' 
constellation and movement [25], and longer sampling dura- 
tion [26]. Most of these investigations, however, do not seem 
to touch upon the rigorous argument on the performance of 
the integer ambiguity resolution, which should be based on 
exact statistical and mathematical discussions. This study sys- 
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tematically discusses the theoretical foundation and efficient 
methodology of the integer ambiguity resolution as formaUzed 
to be an optimal fixed-sample statistical decision, and develops 
the almost rigorous calculation of the maximum-marginal-a- 
posteriori success/error rates under multivariate Gaussian error 
distribution on an integer lattice by using union-bound [27]- 
|29| approximation and importance- sampling Monte Carlo 
integration [30]. 

This study also proposes and examines the two types of 
the enhancement of union-bound approximation of the rates 
for such optimal decisions that are allowed to avoid a judg- 
ment and the optimal decision that addresses a marginalized 
integer lattice. The latter type of decision is applied to the 
succeeding initialization of the integer ambiguity as discussed 
in Section II-D. 

Highly efficient methods are also scrupulously developed 
and applied to the exact calculation of the maximum-marginal- 
a-posteriori and (conditional) Bayes decisions, which are 
required in the process of the Monte Carlo calculations of the 
rates, by using an improved Fincke-Pohst algorithm [31]-[33], 
or sphere decoding. 

This study, then, applies these efficient methodologies to 
calculating the accurate success rates of integer ambiguity 
resolution and reveals the fundamental aspects of the reliability 
performance of the short-period integer ambiguity resolution 
in carrier-phase relative positioning. 

The aspects revealed include the extremely high reliability 
realized by the optimal resolution under the condition of a 
large number of visible satellites, its heavy degradation caused 
by the slight amount of differentiated ionospheric delays due 
to the nonvanishing baseline length of relative positioning, the 
advantages of the use of the multiple carrier frequencies (GPS- 
Ll, L2, and L5 signals are considered in this study), and their 
mutual relations associated with the dependence on the error 
in the carrier-phase and code-pseudorange measurements. 

The performance of the succeeding initialization is shown 
to be greatly improved by the reasonably assumed temporal 
and spatial constancy of differentiated ionospheric delays, 
and, furthermore, by the temporal constancy of the receiver's 
differential clock offset (interfrequency bias) when the contin- 
uous measurement of the attitude of the receiver's antenna is 
available. 

The benefit and trade-off of the well-known wide-laning 
method [34] using the multiple frequencies are revealed under 
the condition of the large differentiated ionospheric delays. 

The only slight difference is clarified in the performance 
between the static and kinematic ("on-the-fly") ambiguity 
resolutions. 

I. GPS/GNSS MEASUREMENTS AND INTEGER 
AMBIGUITIES 

This section describes the statistical formalization of 
GPS/GNSS measurements and integer ambiguities for the 
optimal resolution. 

A. Basic assumptions 

As mentioned, we will concentrate on the fixed-sample 
optimal decision for the integer ambiguity resolution. 



The integer ambiguities involved in the carrier-phase mea- 
surements are assumed to be initially non-informative, and not 
to vary during the resolution for the sake of simplicity. No fail- 
ure in both carrier-phase and code-pseudorange measurements 
is assumed to occur during the resolution. 

All measurements and error variables, including the integer 
ambiguities, ionospheric delays, the (rover) receiver's coor- 
dinates, and the (rover) receiver's clock offset, are assumed 
to have been single-differentiated, i.e., these are the variables 
relative to the reference receiver's ones. We presume to neglect 
the actual slight differences in the measuring and transmitting 
time of the GPS/GNSS signals with regard to the distinction 
between the rover and reference receivers [1], which is justified 
for the theoretical discussions, as this study is. The short or 
medium-range baseline length between the rover and reference 
receivers is assumed, and is specifically within 30 km under 
the conditions of the calculations in Section V-A. 

The GPS/GNSS positioning equations are assumed to be 
linear, which is justified by the fact that the signal waves are 
regarded as plane waves when they are received on the earth. 
GPS/GNSS satellites are assumed to be equivalent in regard to 
both their nominal signal frequencies and the receiver's clock 
offsets. The former equivalence is justified in the case of GPS 
[35], [36] and Galileo [23], and the latter is justified for most 
modern GPS/GNSS receivers. 

Each measurement or prediction error is assumed to be 
distributed according to a Gaussian function with a vanishing 
mean and a certain covariance. Gaussian error distributions 
strongly assume that huge errors or large noises in the mea- 
surements will be relatively rare. Furthermore, a stationary 
random process for the measurement error is assumed, and it 
is modeled as a combination of a temporally constant com- 
ponent and a temporally varying autoregressive (AR) process 
component, the latter of which is assumed to be stationary and 
first-order, i.e., AR(1). 

The a priori distribution of the (single-differentiated) iono- 
spheric delays is also presumed. It is furthermore presumed to 
have a vanishing mean and a certain error variance, the latter 
of which is also presumed to be proportional to the baseline 
length between rover and reference receivers. The (single- 
differentiated) ionospheric delays are assumed to be tempo- 
rally constant during the resolution, and their spatial constancy 
is also assumed when investigating the improved perfor- 
mance of succeeding initialization. Furthermore, the (single- 
differentiated) tropospheric delays are assumed to vanish for 
the sake of simplicity. The (single-differentiated) higher-order 
terms [37], [38] in ionospheric delay are also assumed to 
vanish. These assumptions are satisfactorily justified under the 
condition of the short or medium-range baseline length. 

The satellites' constellation and its movement is calculated 
by using GPS broadcast ephemerides, which are assumed 
to be accurate enough to neglect their coordinate errors for 
the short- or medium-range-baseline relative positioning. The 
sufficiently accurate Une-of-sight directions from the satellites 
to the receiver are assumed to have been calculated by using 
the coarse estimation of the position of the receiver, which 
can be calculated by using the ephemerides and the code- 
pseudorange measurements on the receiver. 
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The boresights of both the rover and reference receiver's 
antennas are assumed to be directed toward the same direction, 
which is likely the local zenith. The attitude of the rover 
receiver's antenna is assumed to be accurately and continu- 
ously measured by using such devices as gyroscopic sensors 
when investigating the improved performance of succeeding 
initialization. The GPS/GNSS electromagnetic signal waves 
[23], [36] are assumed to be exactly right-hand-circularly 
polarized [39]. 

The variation in the phase center of the satellites' and 
receivers' antennas, which is a slight dependence on the 
azimuthal angle of the propagating direction of the GPS/GNSS 
signals and the distinction among their carrier frequencies, is 
assumed to be neglected. 

The signals of satellite navigation are assumed to be GPS- 
Ll, L2, or L5, the carrier frequencies of which are 1.57542, 
1.2276, or 1.17645 GHz [36], respectively. Each carrier fre- 
quency is assumed to correspond to the distinctive signal in 
a one-to-one relationship for the sake of simplicity. Namely, 
the multiplexing of signal modulations, such as that of inphase 
and quadrature component modulations or that of phase-shift- 
keying [29] and (planned) binary-offset-carrier [40] modula- 
tions, is not considered. We assume the receiver's differential 
clock offsets among the distinct kinds of measurements (i.e., 
interfrequency bias) do not vary during the resolution. 

Finally, we assume the initial non-informativeness of 
the (rover) receiver's coordinates, and the initial non- 
informativeness and non-time-correlativeness for the values 
and evolution of the (rover) receiver's clock offsets. Both 
the distinct conditions of the time-constancy and non-time- 
correlativeness of the (rover) receiver's coordinates are used 
for investigating the static and kinematic ambiguity resolution, 
respectively. It is assumed that we are indifferent to the 
determination of the receiver's clock offset in the ambiguity 
resolution. 

B. Equations of GPS/GNSS measurements 

Let i and(t)'^i denote (single-differentiated) GPS/GNSS 



code-pseudorange and carrier-phase measurements, respec- 
tively, where q, i, and k indicate the kind of signals, the 
epoch of measurements, and the distinction of the satellites, 
respectively. They satisfy the following equations [l]-[3]: 
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where ri denotes the coordinates of a receiver. If the iono- 
spheric delay, Tp^q^i and T^,q,i the receiver's code-pseudorange 
and carrier-phase clock offsets (with dependence on the kind 
of measurements), Sp'^ ^ and Scp'^ ^ the code-pseudorange and 
carrier-phase measurement errors, Nq the integer ambiguity, 
and tpi the wind-up phase, and, as already mentioned, they 
are all single-differentiated. ^ and tpi are expressed in a 



unit of cycles. Xq (= c/ fg) denotes the wavelength of the 
carrier wave (and fq is the frequency), denotes the unit 
line-of-sight vector from the satellite k to the receiver, which, 
as already mentioned, is assumed to have been determined 
with sufficient accuracy. is measured as the delay length at 
the frequency of GPS-Ll. 

The (single-differentiated) wind-up phase [41], ^pi, does 
not depend on the distinction among the satellites based on 
our assumption of the equivalence of the rover and reference 
receiver's antermas in regard to their boresight's directions. 
tpi is, thus, equal to the (single-differentiated) right-handed 
rotation angle of the antenna's attitude around its boresight 
direction based on our assumption of the right-hand-circularly 
polarized electromagnetic signal waves of the GPS/GNSS 
signals. 

The kinds of signals, g's, are GPS-Ll, L2, or L5, as 
already mentioned. Two artificially synthesized carrier-phase 
measurements, wide lane (Lw) and extra-wide one (Lew) 
[22], [34], are additionally introduced by using the following 
integer unimodular transformation of the three raw carrier- 
phase measurements: 



(3) 



These carrier-phase measurements should also be accompanied 
by the respective integer ambiguities which are transformed 
by the same matrix as above, q will indicate the distinction 
of these carrier-phase measurements, or that of their carrier 
frequencies, when considering the integer ambiguities. 

For each kind of signals, q, the set of the integer ambiguities 
for all visible satellites, which have been symbolized as k's, 
is linearly decomposed as follows: 





(4) 



where denotes a common integer increment and AN^'s 
the integer ambiguities differentiated among k's. Any vector 
of ANq's is assumed to be perpendicular to that of iV° for 
the sake for simplicity. ANq's are, thus, doubly differentiated 
among the satellites and between the rover and reference 
receivers. Note that the nonuniqueness in the actual repre- 
sentation of ANq's should not affect the optimal ambiguity 
resolution. 

ANq's are decision parameters in the ambiguity resolution, 
whereas Nq is a nuisance one, which only causes the common 
increment to T^,q,i's among i's and to I^'s among i's and k's 
(i.e., their common and temporally constant component for 
each q), and never affects the determination of the positioning 
coordinates, r^, based on our assumption of the equivalence 
of satellites in regard to both their nominal signal frequencies 
and the receiver's clock offsets. 

For each epoch, i, the set of receiver's clock offsets for both 
the code-pseudorange and carrier-phase measurements of all 
kinds of signals, which have been symbohzed as q's, is also 
linearly decomposed as follows: 

T<j>,q,i — ^T<j>,q,i "I" Ti ) 
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where denotes a common increment among all kinds of 
measurements and Ar^ ^ ^'s and Ar^ g ^'s the clock offsets 
differentiated among the distinct kinds of measurements. Any 
combined vector of Arp g ^'s and Ar^ g./s is also assumed to 
be perpendicular to the vector of r". Arp ^ ^'s and Ar^.q.i's 
are also known as receiver's differential clock offsets or 
interfrequency biases [1]. Their actual representations are not 
unique, either. 

Furthermore, based on this study's model, each of both 
kinds of the i-th-epoch measurement errors, Sp'^ ^ and Scj)'^ j, 
is decomposed as follows: 



k 
k 
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where each of 6p'^ q and (5(/)g q is temporally constant compo- 
nent, and each of ASp'^ j and A6(j)'^ ^ is temporally varying 
one modeled as an AR(1) process as already mentioned. Let 
y^ denote the set of all of the i-th-epoch measurements, p^ /s 

denote the 



and (iiig i's, being considered. Then, let y^^^ ^ 
time series of the n-epoch y/s: 



y(l,2....,«) = (yi: ^2: ■ • ■ . Vn) ■ 

The number of epochs, n, is assumed to be fixed in the 
resolution. 

The sets or the time series of the unknown parameters 
or measurement errors that have already appeared will be 
expressed by using the same notation as above. The integer 
ambiguities (0) are then expressed as follows: 



AT = AN + N", 



(8) 



and the time series of the receiver's clock offsets Q and 
measurement errors and are expressed respectively as 
follows: 

T{l,2,...,n) = Ar(i^2,...,n) + (1, 1, • • • , 1)^ X T(l,2,...,n)^ 

^y(i,2,...,n) = 2,....n) + ('5^0' Sy^, Sy^) . 

The statistical likelihood of the set of the unknown parame- 
ters subject to yj]^ 2 n) equals the conditional distribution of 
2/(1 2 n)' which is calculated by using Q and (|3 as follows: 

pi y(l,2,...,«) I T'(1.2,....n), -f(l,2, ...,«), 

"J" (l,2,...,n)7 V'(l,2,...,n)) 

= P(^y(l,2, ...,«)) 

= J p{^^y(i,2,...,n))pi^yo)dSyo- 

The probability distribution of the stationary AR(1) process, 
A5y(^i 2 „), is calculated by using the following equations 
[42], [43]:' 

P(A^y(l,2,...,n)) 

- |^P(A<5y,|A<5y(l 2,...,^-l))| Pi^Sy,), 

A6y,^Ai A(5y,„i +e„ 

where Ai is an AR(1) coefficient matrix and is a station- 
ary error. Note that p{ASyi) (= p{A6y^)) is calculated by 



using (lASy,){ASy,f) ^ {A,y (e, e.^Ai^ , and 

{A6y,) = 0. 

A marginal joint distribution of AN and y^^i 2 «) is. thus, 
calculated by integrating over real nuisance parameters [5], as 
follows: 

p{AN, y(i^2,...,„)) 

^p(y(i.2,....„)|AiV, AfO, r(i,2,...,„), 

-f(l,2,...,n), 'T'(l,2,...,n), V'(l,2,...,n) ) 
X p(A7V, Ar°, r(i,2,...,n), -f(l,2,. ..,«), 

T (l,2,...,n)7 "0(1, 2,. ..,n)) 
X dr(i^2,...,n) dl (1,2.,. ..,n) '^T(l,2,...,n) C?V'(l,2,...,n) , 

where an a priori distribution, 

p{AN, N°, r(i,2,...,„), i'(i,2,. ..,«), 

'''(l,2,...,n), V'(l,2,...,n)), 

is used. 

The marginal joint distribution is factorized without loss of 
generality as follows: 

p(AiV, y(i^2,....n)) 

= p(A7V I iv) xp(y(i_2,. ..,„)), 

and the marginal a posteriori distribution of A7V is expressed 
as: 

p(AiV|y(i^2,...,„))=P(AArk), 

where a minimal sufficient statistic [44], [45], is introduced 
as a function of y^j^ 2 n)' 

i^(y(i,2,. ..,«))• 

1/ is called a floating solution of the integer ambiguity in the 
context of the GPS/GNSS carrier-phase positioning. 

The marginal a posteriori distribution is, thus, calculated as 
follows: 

p{AN, u) 



p{AN I v) 



(9) 



where 



p{AN, u)^^p{u\ AN, TV") piAN, 7V"), (10) 

NO 

and, based on our assumption, the a priori distribution, 
p{AN, TV") (= p{N)), is non-informative, or uniform, with 
regard to both A AT and AT". The summation in the right- 
hand side of dlOt implies that the optimal ambiguity resolution 
should be generally executed on the integer lattice marginal- 
ized over Af° as further discussed in the following sections. 

The general form of the conditional distribution of v (e 
RF), or the likelihood of AT (e Z^), is described as: 

p(i.|Ar) 

where we used our assumptions of linearity and Gaussian error 
distribution and introduced the inverse covariance matrix, M, 
of V. M is calculated by the marginalization over the real 
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nuisance parameters, or practically by using several formulas 
in linear algebra, such as the completion of squares [42], [43]. 

p is the size of u and N, and the following relations 
normally hold: 

p — p^ X number of visible satellites, 
= number of carrier frequencies considered. 

The number of visible satellites is seven and that of carrier 
frequencies is one, two, or three under the conditions of the 
calculations in Section V. 

Let Au and i/" denote the same decomposition of v as that 
of N into A7V and Af" ©: 



u = Av> 



Note that Av is, thus, doubly differentiated as well as A7V. 

If there exists the independence between AN and TV*' in 
the marginal joint distribution, the distribution is factorized as 
follows: 

p(A7V, TV", Au, = p{AN, Au) p{N°, v° \ Au). 

Let us call this case AN -separable in distinction from the 
AN -nonseparable one, the former of which usually arises 
under the condition of a priori non-informative Ar^ ^ 

In the AAT-separable case, the minimal sufficient statistic, 
thus, should not be u, but Au, and the marginal a posteriori 
distribution of AN is calculated as follows: 



p(AiV, Au) 



p{AN\u) 

p(A7V, Au) ^ p{Au I A7V) p{AN), 
p{Au I A7V) 

1 



(12) 
(13) 



' |Ma| 



{Au - AN)' Ma {Au - AN) 



(14) 



where we introduced the inverse covariance matrix. Ma, of 
Au, and pa — P — which is the size of Au and ATV. 

An intermediate case can also occur under the condition of 
a priori non-informative ipi, informative Ar^^q^i, and p° > 2, 
where the size of the minimal sufficient statistic is p — 1. 

Although these kinds of conditions affect the size of the 
minimal sufficient statistic (i.e., the rank of M) and the actual 
calculation in the integer ambiguity resolution, we will use the 
general form (lll> in the following discussion. 

In the calculations, each uniform or translationally invari- 
ant (location-invariant) infinite-interval (a priori) distribution 
should be carefully defined to be the limit of a finite-interval 
distribution (see Appendix I). 

C. Succeeding initialization 

If the correct differentiated integer ambiguity, AAT, 
is once resolved on a multiple-frequency rover receiver, 
the highly accurate estimations of the receiver's 
coordinates, r.^, the ionospheric delays at the location 
of the receiver, J;, and the receiver's differential 



clock offsets, A-Ti, are obtained as their marginal 
a posteriori distribution: p( r(i 2....,n) | AAT, y^j^ 2 n))' 
p{I(i,2,...,n) I A7V, 2 n))' ™d Others (see Appendix II), 
except for the uncertain common increment in li and Ax^ 
among the satellites and the epochs. Such uncertainty in li 
and ATi arises from their coupling to Af", which is left to 
be an unknown integer vector even after the correct integer 
ambiguity is resolved as already mentioned. The a posteriori 
estimation of li and Ar^, thus, distributes multimodally. 

The accuracy of each kind of the a posteriori estimations 
on a single-frequency receiver, however, depends largely on 
the accuracy of the a priori estimation of li which was 
used, which may not be highly accurate except for under the 
condition of the short baseline length of relative positioning, 
as further described in Section V-A. 

Consider the ambiguity resolution on the other receivers 
that reside temporally and spatially adjacent to the multiple- 
frequency receiver with the integer ambiguity already resolved. 
Such resolution should be thus helped by using the obtained 
accurate estimation on li as an a priori one in their resolution, 
based on our assumption of its temporal and spatial constancy. 
Let us call such an initialization the non-self-type of succeed- 
ing initialization. The use of the accurate a priori estimation 
of li remarkably improves the performance in the ambiguity 
resolution even on a single-frequency receiver, as also shown 
in the discussion in Section V-A. 

Furthermore, the receiver, once its integer ambiguity has 
been resolved, gains the potential ability to resolve the possible 
future re-initialization of its own integer ambiguity efficiently. 
Let us call such an initialization the ie//-type of succeeding 
initialization, which is commonly required just after the dis- 
continuity (i.e., the mis-track and re-acquisition) of the carrier 
signals, known as a cycle slip [1], [2], and is frequent in 
the receiver in the mobile environments [4]. This beneficial 
effect arises from the accurate estimation having been obtained 
on both ATi and li, based on our assumption of their 
temporal constancy, as well as the above case. Note that 
the accurate estimation on the receiver's differential clock 
offsets, ATi, is actually achieved only among accurate carrier- 
phase measurements, and thus its noticeable effect is actually 
observed only on a multiple-frequency receiver 

Such estimation and reuse of At.^ is interfered by the non- 
informativeness of the wind-up phase, ipi, i.e., by the condition 
that the attitude of the receiver's antenna is not measured 
continuously. In such a case, three or more kinds of carrier- 
phase measurements are required to achieve the noticeable 
effectiveness of the self-type of succeeding initialization. 

These two distinct, i.e., self- and non-self-, types of succeed- 
ing initialization correspond to the conditions of informative 
and non-informative Ar^, respectively. Each is assumed to 
utilize the respective type of pre-initialization measurements 
with a certain duration and with their integer ambiguities 
correctly resolved. 

II. Optimal success and error rates 

This section describes the optimal ambiguity resolution and 
its success/error rates as formalized to be a fixed-sample Bayes 
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decision. Its optimality was essentially proved by statistical 
studies. A decision, or classification, 6, maps v (or the 
measured sample, y^j^ 2 more general discussion) to a 

value of decision parameter, AN, or avoids a judgment: 



S : 



AN or none. 



The success, error, and reserving rates arisen from S are 
calculated by integrating the marginal a posteriori (joint) 
distribution (I10> as follows: 



AN AN' -^DsiAN') 

75 = 1 - - (3s, 

in which Ds{AN) is defined to be the acceptance region 
where the decision 6 chooses ATV in the space of i/. 

These rates are calculated by using (lll> and our assumption 
of the non-informativeness of p{N), a priori distribution of 
N (= AN + N°), as follows: 



as 



iDsiAN) 



p{ i^\AN, N")du 



cxp<j --{v- NfM {t>-N)\ du, 



and 



Ds{AN) 



^ / p{,.\AN,N")di^ 

^j^, JDs{AN'} 
=^AN 



(15) 




Ny M{u~N) }du 



(16) 

where the independence of as and Ps on AN and AT^ 
is used. Consequently, the integration domain in ( II6I 1 is 
UAJVVAAr^'5(A-/V') with regard to v. 

A maximum marginal a posteriori decision, (5map, is de- 
fined by using the marginal a posteriori distribution as: 



Smap ■ f 
AATmap = 



ATVmap, 
argmax p{AN \ u) 

AN 



(17) 



and never avoids a judgment. The decision is well known 
to maximize as subject to 75 — 0, and this optimality was 
essentially proved by [46]'. Its unconditionally optimal rates, 
ttMAP and /3map, should be calculated by the integrations (I15> 




(a) AAT-separable case in the space 
of Au 




(b) AJV-nonseparable case in the 
space of u 



Fig. 1. Two-dimensional error distributions and the acceptance regions of 
_DiviAp(^-^) and D{AN; h) for success/error rates. 



and M6\ with ^map applied. Its acceptance region is denoted 
as i:'MAp(AAf). 

A conditionally optimal decision, 5opt, is defined as the 
modification of (5map: 



Oopt 



I AATmap, if p(A7Vmap \i^)>h; 
I none, otherwise, 



(18) 



in which a Bayesian confidence parameter, h (> 1/2), is 
introduced, and the case of equality is incorporated with the 
greater case for the sake of simplicity. (5opt is also well known 
to maximize as subject to Ps < Ph with f3h assigned, and this 
optimality was essentially proved by [47]^ (see [44], [45]). 

Its conditionally optimal rates, aopt, Popt, and 7opt, should 
be also calculated by the integrations il5\ and (I16> by applying 
(5opt and choosing the value of h so that /3opt should equal 
Ph, assuming that (3h is assigned to be less than Pmap- Or 
else, (raw) (5m ap should be used, and then aopt = auAP, 
/3opt — Pmap, and 7opt = 0. Let us explicitly denote the 
dependence on h of Dopt(AAr), aopt, Popt, and 7opt, and 
omit their subscript, "opt," in the following discussion when 



This was rediscovered by [6] in the field of GPS/GNSS. 



^This was also rediscovered by [48] in the field of GPS/GNSS. 
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Fig. 2. One-dimensional error distribution and tlie acceptance regions for 
success/en'or rates. 



they will not be misunderstood, as follows: D{AN; h), a(h), 
f3{h), and -f{h). 

The following approximate equations hold: 

D{AN;y2)~ DuApi^N), 

a(V2) ^ ttMAP, 
(3(1/2) ~ /?MAP. 

The difference between the right-hand and left-hand sides 
of each approximate equation is extremely small under the 
condition of the high success rate of omap, or the low error 
rate of /3map- 

There exists a slight probability that the following inequality 
holds: 



p(AiVMAP I I^) < 1/2- 



(19) 



This probability is extremely low under the condition of the 
extremely high success rate of Q!map- 

Furthermore, let us normalize h and introduce h' so that 
the maximum of the marginal a posteriori distribution exactly 
corresponds to h' = 1, as follows: 

h 



h' 



ha ' 



ho = piAN I u = AN + N°) ~ 1. 

In the following discussion, the normalized h' is used instead 
of the non-normalized h. The functions of success, error, and 
reserving rates will be represented as functions of h', such as 

D{AN;h'), a{h'), f3{h'), and -f{h'). 

III. The methodologies of the calculations of the 

rates 

This section explains the methodologies of the calculations 
for the optimal success/error rates of integer ambiguity reso- 
lution. They are basically a union-bound approximation and a 
Monte Carlo integration. 

A. Union-bound approximation 

1} c^map md /3map in the AN -separable case: Let us 
firstly consider the calculations of ajviAP (I15> and /3map (I16> 
in the AAT-separable case. Note that the actual integrand is 
(I14t using A/a and the integration variable is Au in this case. 
Generally, if the marginal joint distribution is a mono- 
tonically decreasing function of the positive-definite quadratic 



form of A7V — Au, the acceptance region, Dmap{AN), is 
shaped into a uniform point-symmetrical Voronoi polytope in 
the space of Av>, as shown in Fig. |l(a)| Each lattice point, 
A7V, resides at the center of the polytope. 

Voronoi-relevant, i.e., adjacent lattice points, are defined, 
as their distinct Voronoi polytopes share facets. The adjacent 
lattice points can be represented as displacement vectors from 
a certain lattice point to its adjacent ones. Let us denote the 
set of adjacent integer lattice points as 



{ni, n2, ■ • • , rii 



(20) 



which contains q lattice points. This set contains all possible 
lattice points, but any two of them, and rij, should satisfy. 



\/ u^'^Ma rii 



(21) 



in which the generalized distance, a^, of each adjacent lattice 
point from the origin is introduced. The representation of j20t 
will not be assumed to distinguish between point-symmetrical 
lattice points, i.e., rii and — n^; thus, they are counted as one 
lattice point in ( I20> . 

Each adjacent lattice point, rii, corresponds to a facet of the 
Voronoi polytope, which is a generalized perpendicular bisec- 
tor of rii. At the bisection point, the following approximate 
equality holds under the high success rate and low error rate 
conditions: 

p{AN I Au = AN ± 1/2 n,) ~ 1/2- 

The number of facets is 2q, and q is bounded by the inequality 
[49], 

If Ma is a diagonal matrix, it produces a rectangular Voronoi 
polytope and q equals p^. For a general matrix, however, q 
usually takes on its maximum value, 2^^'^' — 1 [50]. 

It is well known that the union bound [27]-[29] is the lower 
one to CKMAP (and the upper one to /3map), and the minimum- 
distance bound [51], [52] is the upper one to omap (and the 
lower one to /3map), as follows: 



ttMAP,] 



> "MAP > 1 - /?^ 



W QfMAP.i, 
i=l 



q 



/3map .mill < /3mAP < /^MAP,! , 
i=l 



(22) 
(23) 
(24) 



2a, fV^ / 1 2 2\ 
aMAP.i = -^j^ J exp|^--ai x j dx, 

li f°° / 1 2 

= J exp^--ai X j dx = 1 - aMAP,i, (25) 



aMAP.min 



2flmin 

V2^ 



V2 / I 

cxp -Tramin^a;^ ) dx, 







/^MAP.min — niax/^MAP.i — 1 ^ aMAP,i 

i 

ftmin = mina;. 



g 



2) a{h') and (3{h') in the l^N -separable case: Secondly, 
let us consider the calculations of a{h') and f3{h') in the 
ATV-separable case. The shape of D{AN; h') remains as the 
incomplete Voronoi polytope whose pseudo-facet is tangent to 
the ellipsoidal equivalence surface of the function (I14> shown 
in Fig. |l(a)| based on our assumption of the high success rate 
and low error rate conditions. 

Then, let us introduce one-dimensional a posteriori distribu- 
tions, each of which will determine the position of the pseudo- 
facets, as follows: 

exp{-ia,2(n-^)2} 



Pi{n\£,) 



Its normalization factor is 




1 2 /2 

expl ——CLi n 



and the value at the bisection point is 

Pi{n\£,=n± 1/2) ~ 1/2- 

The last two approximate equalities are derived from the same 
assumption as mentioned above. Fig.|2]shows one-dimensional 
error distribution, Pi{i\n — 0), one-dimensional a posteriori 
distribution, pi{n\^), and the acceptance regions in the space 

We propose the approximated bounds to a{h') and (3{h'), 
which are the variants of the union and minimum-distance 
ones, as follows: 



(26) 



/9min(/i') < f3ih') 



< 



q 



i=l 



2a,: 



2a 



a^{h') = 

= 

\J l-K JO 

/3min(/i') = max/3,;(/i'), 



M 2 2 

expl "2"* ^ 



exp[ -]^ai^x^ 



dx, 



(27) 

(28) 
(29) 



exp 



2 2 



dx. 



where should be calculated by using 

p,(o|6) = /i'V», 

with h' assigned. These formulas coincide with J22> - J25> by 
substituting = 1/2. The right-hand side of (I27t gives the true 
upper bound to l3{h') in the ATV-separable case, whereas the 
right-hand side of i26\ is expected to give the approximate 
lower bound to a{h'). 

Let us consider the functions of x, a{x) and 1 — /3(1 — 
x), which are approximately bounded by the above formulas. 
They approximately coincide in the neighborhood of a; = 1/2, 
because of the following small one-dimensional error rate at 

= V2 : 

= 1/2) = = 1/2) « 1. 



3) Success/error rates in the AN-nonseparable case: 
Finally, let us consider the calculations of q:map, Pmap, 
a{h'), and P{h') in the ATV-nonseparable case. In this case, 
the marginal a posteriori distribution (|9} is expressed by using 
AI (not Ma), whose acceptance region in the space of i/ 
(not Ai/) is shaped as shown in Fig. |l(b)| We can define 
adjacent displacement vectors, rii's, as well as i2Q\ . but by 
using M instead of Ma- Thus, we propose to use the same 
approximated bounds as ( I22> -( I25> and ( I26> -(I29> in this case, 
with the proviso that we should use such adjacent lattice 
points, n/s, that are not any possible vectors of iV°'s: 



{ni, n2, . . . , rii, . . . , n^/} , = n. 



(30) 



Their adjacency condition requires that any two of them should 
satisfy. 



^rii^M rij I < fli^, 



(31) 



as well as \2\\ . 

This set exactly provides the set \2Ql derived from A/a, if 
we transform M to a AAf-separable matrix. (Consider, for 
example, the transformation of the condition of Fig. |l(b)| to 
that of Fig. El) 

4) Modifications: Although the approximated union bounds 
being proposed are expected to be tight under the high success 
rate and low error rate conditions, their tightness is degraded 
by any condition that lowers the success rate a{h') by increas- 
ing h' . The approximated minimum-distance bound proposed 
for /3(/i') may, furthermore, fail to bound the exact value under 
such conditions, especially in the AAT-nonseparable case. 

In order to improve their accuracy under such conditions, 
we furthermore propose to modify both the bound to error rate 
\21\ and the one-dimensional error rate \29\ into 

qor q QOr q 

PnUh') n "'('^') < /^(^') ^ J2 ^^(^')' (32) 



1=1 
2ai 



Mh') = ^ exp 



V27r 

Pminih') = max/3,,(/i') 



'-ai^x^ I dx, 



(33) 



These modifications, (j^D and j33L will be used instead of \21l 
and \29\ in the following calculations. In general, an excess 
degeneracy of the polytope's facets closer to the center point, 
or a large number of the adjacent lattice points with small ai, 
may degrade the accuracy of the union and minimum-distance 
bounds. 

If the single approximation formulas are preferred instead 
of the bound ones, we propose to use the following approxi- 
mation for (3{h'): 



i=l j=l 

and the lower bound in (I26> for a{h'). 



(34) 
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B. Importance-sampling Monte Carlo integration 

The approximated union bound proposed in Section IV-A 
is not always tight enough to investigate the rigorous value 
of QfMAPj Pmap^ C({h'), and f3{h'), even under the high 
success rate and low error rate conditions. A numerical Monte 
Carlo method [30], by contrast, always provides the reliable 
calculation of the rates, but suffers from the severe slowness of 
numerical convergence in high-dimensional integrations. This 
study will accelerate the convergence by using the importance 
sampling method [30] described below. 

7 ) Basic formalization: Consider the integration of multi- 
variate Gaussian distribution inside a certain closed domain, 
D, as follows: 



where p is the size of the inverse covariance matrix, AI. I 
is assumed to be as in (I15> or (3s in il6\ with x set to 
V — N and D is, thus, assumed to be Ds{^N = 0) or 
[J^^,_^fy Ds{AN'), respectively, with regard to x. 

This study also assumes that random points, ^^'s, are 
generated for a Monte Carlo integration for / with regard to 
X and distribute according to distinct multivariate Gaussian 
distribution with the inverse covariance matrix, M : 

The hit-or-miss Monte Carlo integration [30] for / is, thus, 
described as: 

where N is the total number of generations and a "0-1" reward 
function is used: A{V) = 1, if 7^ is true; 0, otherwise. This 
study uses the reliable algorithm [53] for the generation of 
random numbers. 

The coefficient of the square of the error variance, k, is 
defined and calculated as follows: 

K = ^lim (^(^In - X N 

= J / eiipl--x^(2M ~ M)x\dx, (37) 

V (27r)P|M| Jd \ "2 ^ ) \ ' ' ' 

where we used the assumption that the number of the drops 
of ^j's in D is much less than N . This assumption is justified 
if we consider (3s as / under the high success rate and low 
error rate conditions. 

We numerically optimize each element of M to minimize 
K for (3{h') by using the Broyden-Fletcher-Goldfarb-Shanno 
method [54], [55], where the integration part in (|^ is 
normalized as: 



and is calculated by using ( I34L in which ai{h') and (3i{h'), 
i.e., M%\ and ( I33l l. should use distinct a\ instead of a, in ( 13 11 1 
as foUows: 

J njM {2M - Af) M n, 

2) Another type of formalization: Another type of the 
distribution for importance sampling accelerates the Monte 
Carlo integration for (3{h') with large h'. It is derived from 
the following modification of the integration: 




(38) 

where x is assumed to be — TV + N' in ( I16t . and thus the 
integration domain is D{AN: h') with regard to x. We also 
assumes random points, ^/s, are generated for x and distribute 
according to the same form of multivariate Gaussian distri- 
bution with the inverse covariance matrix, M, as (l36ll.This 
type of the importance-sampling Monte Carlo integration for 
( I38> with optimized M provides the faster convergence for the 
integration of (3{h') in the region of about logj^Q h' /{\ — h') > 
15 in the calculations of Section IV-D, whereas the former 
type of the optimized Monte Carlo integration for i35\ suffers 
from the slowness of the convergence in that region. 

C. Efficient calculations of the optimal decisions 

This subsection explains how to improve the efficiency in 
the critical parts of the calculations for the optimal decisions in 
this study. The first half introduces the basics of the algorithms 
and the last half proposes their enhancements required in this 
study. 

7 ) Sphere decoding: We use the efficient algorithm for the 
search and the enumeration in an integer lattice involved in the 
optimal decisions, which was essentially developed by Pohst 
and Fincke [31], [32], and is known as a sphere decoding, and 
has been applied to such problems as a closest vector one, a 
shortest vector one [33], and a communication decoder [56], 
[57]. It is the branch-and-bound-type search [58], [59] down 
to the bottom layer of the search tree, which enumerates the 
lattice points, TV, enclosed by a p-dimensional base ellipsoid. 
This corresponds to the inequality for the quadratic form using 
V and M in (II 11 1, as follows: 

(TV - vf M (TV - u) 

- (TV' - u'fl'^L' (TV' - I/') < x', ^^^^ 

where 

L'^L' = M' = {Z-YmZ-\ 

N' = ZN={N[, TV^, TV;)^, 

u' = Zu^ [v[, v'2, v'p) , 

and the "radius," of the base ellipsoid is introduced. M is 
preprocessed to Al' by using the transformation with an integer 
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unimodular matrix, Z, in order to reduce the lattice basis ahead 
of the actual search as further described in the paragraph after 
next. The lower triangular matrix, i', is calculated by using 
the Cholesky decomposition of M' . 

The search is comprised of the self-homogeneous searches 
for the sub-dimensional lattice points enclosed also by an el- 
lipsoid. The base inequality j39> is, consequently, decomposed 
into the following sequential inequalities: 



< 



< 



(2) 



(2) (-^(2) 



T' '^T 



'(1) 



(2) 



(P) 



(40) 



in which we 



submatrix. 



N' 



have 

T 



introduced subvectors, TV 



(■0 



and = 



and a 



^21 



(i) 







where i = 1, 2, . . . , p. 

In each layer in the downward process of the search, any 
of the incomplete (or complete if i — p) solutions, AT'^j^'s, 
should satisfy the following inequality: 



T' 



The number of the solutions, AT^j^'s, is, consequently, roughly 
estimated to be 



.V2 



r (V2 + 1) 



i=i 



The numerical complexity of the search in the full, or p-, 
dimensional space is, thus, estimated to be 



p 

E 

i=l 



rji or 



(41) 



The preprocessing reduction of the lattice basis deforms 
the base ellipsoid in order to diminish such numerical com- 
plexity by applying the transformation by Z. The celebrated 
algorithm of the lattice basis reduction introduced by Lenstra, 
Lenstra, and Lovasz [33], [60] efficiently provides the rea- 
sonably descendingly -reordered^ diagonal elements, ij^'s, and 
is widely used in the implementation of sphere decoding as 
well as in the original one by [32], and is also used in the 
calculations in Section V-A. 

'Actually, the original formalization by [60] reorders ascendingly the 
diagonal elements of the upper triangular matrix derived by the Cholesky 
decomposition, M' = V'^U', which is commonly used by the implementa- 
tions of sphere decoding such as [32], [33]. 



A suboptimal solution, TVg = (^N^ ^ , TVg 2 7 



is defined as each of its elements being derived by minimizing 
each side of the inequalities ( I40> sequentially down to the 
bottom layer. This solution was introduced by [61] and is 
known as a Babai nearest plane solution"*, whose complexity of 
calculation belongs to the polynomial-time class with regard 
to p. This solution itself is statistically inadmissible^ and is 
not used to calculate the success/error rates in this study. It is 
used to bootstrap the efficient algorithm described below. 

2) The Schnorr-Euchner strategy with the dynamic shrink 
of the radius: The search algorithm is furthermore improved 
by using the following refinements developed by [56], [57], 
[65]-[67], which are also used in this study. They use the 
depth-first movement [58], [59] in the search tree, observe the 
distancing order with ATg as starting point for the branches 
in each layer of the search tree [65]: 

Nl = K,, Ni,^, + 1, Ni,^, - 1, Ni,^, + 2,..., 
or = K,, Ni,^, - 1, 7V^,,; + 1, 7V^,^ - 2, ... , 

and dynamically and rapidly shrink the radius, x> each time 
the movement succeeds in reaching a complete solution, TV', 
residing in the bottom layer in the process of the search. 

For a closest vector problem or a shortest vector problem, 
such update of x is calculated as follows: 



(42) 



which is also used in the calculation for the maximum 
marginal a posteriori decision ( I17> in the ATV-separable case. 
TVg is, consequently, the first reached complete solution in 
this refined algorithm. It actually defines the initial radius, xb, 
according to Hli . and, thus, determines the actual numerical 
complexity of the search based on the estimation J4U with xb 
substituted in 77^. 

These algorithms are used in this study to make the maxi- 
mum marginal a posteriori decision illi and the conditionally 
optimal decision ilEl . both of which are included in the 
laborious Monte Carlo calculations for the optimal success 
and error rates. These algorithms are also used to construct the 
set of Voronoi-relevant lattice points, i20\ or i30\ . Both types 
of calculations are known to belong to the nondeterministic- 
polynomial-time class with regard to p. The latter calculation 
furthermore uses the algorithm introduced by [67], which 
efficiently examines Voronoi-relevant lattice points, n/s, by 
using the following equation: 

|TVe Z'P; {N - y2nif M {N - y2ni) < 1/40,^} 
-{0, nj. 

In order to reduce the lattice basis by using Z in the 
step of the preprocessing, we use the Lenstra-Lenstra-Lovasz's 



^It is also known as a sequential-rounding resolution, which has been 
rediscovered and used in the field of GPS/GNSS [62]-[64]. 

'in [6], the admissibility of statistical decisions or classifications was 
misdefined in the discussion about several kinds of truly invariant integer 
ambiguity resolutions. 
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Fig. 3. The rates calculated by using Monte Carlo integration and approxi- 
mated bounds in the AAf-separable case. 



Fig. 4. The rates calculated by using Monte Carlo integration and approxi- 
mated bounds in the AAf-nonseparable case. 



algorithm with its S parameter set to one^, whose condition 
has not yet been proved to belong to the polynomial-time 
class with regard to p, unlike the conditions of 1/4 < <5 < 1. 
The reordering procedure built in the algorithm of lattice basis 
reduction should be restricted inside each subspace of AN and 
N'^ in the AAT-nonseparable case because of the necessity to 
calculate the marginal joint probability ilOi in the pA-th layer 
in the process of the search. 

The extra column-reordering introduced by [32] is not used 
in this study because it is examined and does not reduce the 
estimation of the numerical complexity ( I41> . The strategy of 
the lattice basis reduction applied to M^^ instead of M also 
introduced by [32] is not used either for the same reason. 

This study efficiently calculates the infinite sum appearing 
at the denominator (and numerator) in the marginal a pos- 
teriori distributions, and J12l i. by using the finite, i.e., 

^In [12], the same condition was used accompanied by both rediscoveries 
of this algorithm for lattice basis reduction and the Fincke-Pohst algorithm for 
a closest vector problem. In the field of GPS/GNSS, the most methods (see 
[7], [13], [14]) oriented to the efficient calculation of the integer ambiguity 
resolution indeed use these algorithms inspired by [12]. 



approximated, set of TV's. This set excludes such TVs that 
only contribute extremely slightly to the sum, and is actually 
constructed by using the sphere-decoding search with its 
bounding radius, x, dynamically updated as follows: 



X 



{N- 



N - 



where a constant, c, is added by this study. This approximation 
is justified when we choose a sufficiently large value for c 
enough to neglect the truncation error induced. 

The similar approximation also improves the efficiency in 
the calculations of the union bound, ( I20> -( I25> . ( I26t -(l29ll. and 
(I30> - (I33> . or that of the Voronoi-relevant lattice points, by 
using the approximated set of a/s which is produced by elimi- 
nating such Qi's as larger than given ath. This approximation is 
justified as well by the fact of the extremely slight contribution 
of the union of such large a/s to the total sum or product for 
the union bound, when we choose a sufficiently large value 
for Oth compared with a^in- This is also actually realized 
by using the sphere-decoding search with i/ set to and its 
bounding radius, x (= ath)> dynamically updated each time a 
new Voronoi-relevant lattice point, n^, is found in the process 
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TABLE I 

The distinct sets of GPS measurements used in the 

CALCULATIONS. 



symbol 


the set of measurements 


LI 


4>u and PLl 


L1+L2 


4>LU 


<Al2. Pli. and pL2 


L1+L2+L5 


<t>Ll, 


't>L2, <t>L5, PLl, PL2, and PL5 


Lw 




PLl, and PL2 


Lw+Lew 


4>L^, 


"/"Lew PL I. PL2, and pLs 



TABLE II 

Individual conditions of calculations. 



con- 


variance of error 


in single-differentiated 


dition 


code-pseudorange 


measurements 




• time-varying 


• time-constant 




component 


component 


(a) 


0.5 m 


0.5 m 
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variance of single-differentiated 
ionospheric delay, cr^j (irvm) 



(a) condition (a) in Table II 



of the search, as follows: 



c, 



where a constant, c, should be also chosen to be sufficiently 
large. 

D. Calculation test 

We calculate the accurate success/error rates by using the 
Monte Carlo integration and examine the tightness of the 
proposed approximated bounds by comparing them. 

a{h') and (3{h') are accurately calculated by the accelerated 
Monte Carlo integration under two distinct conditions, which 
are shown in Fig.|3land|3 respectively, accompanied by those 
calculated by the approximated upper and lower bounds i26\ . 
( I28> . i32\ . and (I33> . The former condition is chosen in the 
AAf-separable case and the latter in the AAT-nonseparable 
case. The details of both conditions are carefully chosen based 
on preliminary surveys and they have comparably high auAP, 
i-e., logio aiviAp/ (1 - "map) ~ 6.5. 

In Fig. |3] and |4] the (exact) q:map and /3map are cal- 
culated by the Monte Carlo integration and shown as the 
lowest ends, or the left ends, of the plots in the region of 
login 'i'/ll - h') < 0, or p(A7Vmap| i^) < V2- The almost 
horizontal section of each plot in this region indicates its slight 
probability of occurrences, as mentioned in ( I19> . Note that 
ttMAP ~ "(1/2), /3map ~ /3(l/2), and omap = 1 - Pmap- 
The plots of a{h') and P{h') should be looked at in the region 
of logj^o ''■'/(I ~ h') > to discuss the tightness of the bounds 
in the following discussion. 

The results qualitatively examine the tightness of the bounds 
to aMAP, which provide logio aMAp/(l — "map) = 6.5008, 
6.5007, and 6.5525 by the Monte Carlo integration, the union 
bound, and the minimum-distance bound, respectively, under 
the former condition, and 6.5410, 6.4319, and 6.7231 under 
the latter condition. 

The tightness of a{h') and f3{h') in the region near 
logj^Q — /i') = is almost similar to that of Q!map and 
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Fig. 5. The dependence of the success rates of integer ambiguity resolution 
on the uncertainty of ionospheric delay. 



pMAP under each condition. The approximated union bound 
under the former condition is extremely tight in the region 
of log]^Q/i'/(l — h') < 15. This is regarded as common in 
this case under the condition of high q;map- Under the latter 
condition, however, the approximated union bound, as well 
as the minimum-distance bound, is not extremely tight even 
in the region near logj^p h' /{I — h') = 0. We thus conclude 
the (approximated) union bound does not always provide its 
excellent tightness in the AAT-nonseparable case even under 
the condition of high omap, and it should not be used as 
the approximation of the rates in such a case if we want to 
analyze the performance of the integer ambiguity resolution 
rigorously. 
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IV. Applications to the analysis of the integer 

AMBIGUITY resolution 

This section analyzes how to reaHze the high reliability of 
the short-term integer ambiguity resolution in carrier-phase 
GPS/GNSS positioning by calculating the optimal success 
rate, auAP, and its dependence on such conditions as the 
uncertainty in ionospheric delay, the initialization time, and 
the difference between the static and kinematic resolutions. 

A. The dependence on ionospheric delay and initialization 
conditions 

1) Calculation of auAP- Fig. |5l shows the dependence of 
ttMAP on the error variance of a single-differentiated iono- 
spheric delay, aAi- Their upper abscissa indicates the baseline 
length which is simply estimated to be u^/ multiplied by 10^. 
Table I shows the five distinct sets of code-pseudorange and 
carrier-phase measurements used in the calculations. Note that 
the set of Lw or Lw+Lew artificially excludes the inherent 
equation of LI carrier-phase measurement from the set in (|3jl. 
The precise positioning should be achieved by resolving the 
integer ambiguities involved in L1H-L2 or L1h-L2h-L5 mea- 
surements, or by resolving those in any sets of measurements 
under the condition of aAi as further discussed later. 

The calculations also assume two distinct kinds of con- 
ditions (a) and (b), which have a difference in error in the 
code-pseudorange measurements, and correspond to Figs. |5(a)'| 
and |5(b)| respectively. The distinct part of these conditions 
is described in Table II, and the common part is described 
in Table III. The calculation is executed under the condition 
of a priori non-time-correlative receiver's coordinates, i.e., 
kinematic (on-the-fly) resolutions. 

The variances of the two kinds of error components in 
measurements, i.e., a temporally constant one and a temporally 
varying one, are always set at the same value as each other, 
as in Table II. This means the magnitude of the total error 
variance is equal to that of the individual one multiplied by 
V2. 

Fig. |5(a)| furthermore contains the plots calculated under 
the condition of self- and non-self-succeeding initialization 



TABLE III 

Common parameters of the calculations. 



number of GPS satelHtes 
variance of eiTor in single-differenti- 
ated carrier-phase measurements 

• time-varying component 

• time-constant component 

AR(1) coefficient in time- varying 
component 

• carrier-phase measurements 

• code-pseudorange measurements 
measurement rate 
measurement time 
pre-measurement time 

in the succeeding initialization case 

a priori prediction of receiver's 
coordinates and clock offset 

a priori prediction of receiver's 
interfrequency biases 
a priori prediction of single- 
differentiated ionospheric delay {<jai) 

a priori prediction of single- 
differentiated tropospheric delay 



0.02 cycle 
0.02 cycle 



0.95 
0.5 

1 epoch/sec 
10 sec 
10 sec 

non-informative and 
non-time-correlative 

non-informative and 
time-constant 
predicted with uncer- 
tainty and time-con- 
stant 

(deterministic) 



using the measurements of LI, L1H-L2, or L1h-L2h-L5. Their 
measurement time for the ambiguity resolution is assumed 
to follow the pre-measurement one without interval. The 
condition of self-succeeding initialization is also assumed to 
be informed of the accurate values of the phase wind-up by 
measuring the attitude of the receiver's antenna continuously. 
Otherwise the success rates is examined to be degraded to the 
value close to that under the condition of a non-self-succeeding 
initialization. 

Each plot of q:map is essentially calculated by using the 
Monte Carlo integration. The series of the dense points of 
the union-bound solutions are also calculated and overwritten 
on any sections of the plots where the difference between 
these two kinds of solutions is assessed as extremely slight 
or undetectable. The points of the Monte Carlo solution 
are not densely plotted in such sections in order to save 
total computation time, which is justified by the reasonable 
assumption of the intrinsic smoothness of the plots. 

The whole of a plot, in contrast, is calculated only by using 
the Monte Carlo integration in the cases of the small value 
of omap (i.e., logioaMAp/(l - "map) < 2) and in the 
case of the non-self-succeeding initialization using the L1H-L2 
measurements, where the union bound is not tight to the exact 
value and should not be used to plot any of the points. It 
consumes huge computation time to calculate the whole of a 
smooth plot, i.e., the accurate series of the dense points, by 
using the Monte Carlo integration, if it has the high value of 
aMAP (i.e., logio aMAp/(l - Q;map) > 4). 

2) Results and considerations: Figs. |5(a)| and |5(b)| indicate 
that the high success rates are achieved at a/i/ = 0, i.e., under 
the condition of a vanishing baseline length, especially by 
using the L1h-L2h-L5 measurements. The use of the L1H-L2 
measurements provides the high success rate only under the 
condition of a vanishing baseline length and small error in 
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code-pseudorange measurements shown in Fig. |5(a)| The large 
error in code-pseudorange measurements noticeably degrades 
the success rates except for in the case of L1+L2+L5 or 
Lw+Lew measurements, shown by comparing Figs. |5(a)| and 
15^5)1 

The increase in a/i,j severely degrades the success rate 
especially under the condition of the use of L1+L2+L5 or 
L1+L2 measurements, which contrasts with that of Lw or 
Lw+Lew measurements. This proves that the precise position- 
ing essentially suffers from the low reliability of the integer 
ambiguity resolution under the condition of a long baseline 
length. The Lw or Lw+Lew measurements have the disadvan- 
tage in the precision in positioning under this condition as 
further discussed below. 

The use of the succeeding initialization almost overcomes 
the a priori uncertainty in ionospheric delay by actually 
obtaining the accurate estimation of <jai, which is derived 
from the use of the pre-initialization measurements with their 
ambiguities correctly resolved. The self-succeeding initializa- 
tion furthermore almost overcomes the a priori uncertainty 
in the differential clock offset of the receiver and gains the 
increase of the success rate, shown by comparing it with the 
case of non-self-succeeding initialization. 

3) Precision in positioning: Fig. |6l shows the dependence 
of the variance of range errors on that of the error in single- 
differentiated ionospheric delay for each kind of measurements 
being analyzed in this study. Each range-error variance corre- 
sponds to the least measurement error achievable in the geo- 
metrical range, which corresponds to a posteriori distribution, 
p( ''"(i,2....,?j) I ^-^1 2/(12 n))' ^Iso Appendix II). It is 
calculated under the condition (a) in Table II. This inves- 
tigation is important because the actual precise positioning 
should be evaluated from the perspectives on its attainable 
geometrical precision in combination with the performance of 
integer ambiguity resolution. 

The error variances attained from the LI, L1+L2, and 
L1+L2+L5 measurements retain the original precision in 
carrier-phase measurements and achieve the high precision 
in the geometrical range except for the LI measurements 
under the condition of large a^i. Those attained from the 
Lw and Lw+Lew measurements, by contrast, lose the original 
precision in carrier-phase measurements and have a larger 
range error than that from the LI measurements. This reveals 
the trade-off in the use of these artificial measurements (Lw 
and Lw+Lew) between the precision in positioning and the 
performance of integer ambiguity resolution under the condi- 
tion of large aAi- 

B. Temporal variations 

We investigated the dependence of the optimal success 
rate on both time and sampling duration, the latter of which 
is the initialization time consumed in the integer ambiguity 
resolution. 

Figs. |7(a)| and |7(b)| show the temporal variations of the 
success rate, omap, under the conditions of the use of 
L1+L2+L5 and L1+L2 measurements, respectively, and under 
the distinct conditions of the sampUng duration, which is set 
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Fig. 7. Temporal variations of the success rates of integer ambiguity 
resolution under the condition of several sampling durations and the use of 
two distinct measurement sets. 



to 1, 10, 25, or 50 seconds and depicted as horizontal bars 
of corresponding length. The satellites' constellation and its 
movement is derived by using a GPS broadcast ephemeris 
in this calculation. The error variance of single-differentiated 
ionospheric delay, u^/, is set to 5 mm and other parameters 
are set to the same as those under the distinct conditions (a) 
in Table II and the common ones in Table III. 

The two distinct conditions of the time-constant receiver's 
coordinates (static ambiguity resolution) and the non-time- 
correlative ones (kinematic one) are additionally examined and 
depicted in each figure except for the sampling durations of 
one second (note that the measurement rate is 1 epoch/sec in 
this calculation). Each plot of q;map is essentially calculated 
by using the Monte Carlo integration. 

The results indicate that the constellation of the satellites 
affects QfMAP whose temporal variations are estimated to have 
almost the same amplitude of about 0.5 in logj^Q aMAp/(l ^ 
ttMAp) for each duration, appearing in the whole calculated 
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time (300 seconds). The rate grows with a longer sampHng 
duration, and, furthermore, the growth is accelerated by the 
constellation's movement within the duration. The acceleration 
effect is, however, not temporally homogeneous (compare the 
plot of the 50-second duration with the one of 1 -second, in 
regard to the degree of decrease after 200 seconds in both the 
figures). 

The effect of the satellites' constellation largely affects 
Q!MAP especially under the conditions of the use of L1+L2 
measurements and the short (1 -second) sampling duration 
shown in Fig. |7(b)| which contains the short-term degradation 
in the success rate (compare the plot at about 120 seconds 
with the one under the use of L1h-L2h-L5 measurements in 
Fig.|7(a)}. 

The static ambiguity resolution has slightly higher success 
rates than those of the kinematic one, as shown in both the 
figures. This is caused by the constellation's slight movement 
during the ambiguity resolution. The effect is not temporally 
homogeneous either, as also shown in the figures. 

V. Conclusion 

This study developed the almost rigorous calculations of 
the maximum-marginal-a-poifeWoW success rates of statistical 
decision for multivariate Gaussian error distribution on integer 
lattices, and applied them to revealing the various possible 
realizations of the reliable and short-period integer ambiguity 
resolution in carrier-phase relative positioning by GPS/GNSS. 

The extremely high reliability was shown under the con- 
ditions of accurate multiple-frequency carrier-phase measure- 
ments and a large number of visible satellites. Its heavy 
degradation was revealed under the conditions of the even 
slight amount of differentiated ionospheric delays originating 
from the nonvanishing baseline length between the rover 
and reference receivers. The succeeding initialization of the 
integer ambiguities was shown to overcome such a disad- 
vantageous condition remarkably due to the temporal and 
spatial constancy of differentiated ionospheric delays. The 
shght difference was also shown in the performance between 
the static and kinematic short-period ambiguity resolutions. 

This study makes it possible to investigate the achievable 
performance of the short-term integer ambiguity resolution in 
the field of GPS/GNSS rigorously. 

Appendix I 

A uniform, or translationally invariant (location-invariant), 
infinite-interval (a priori) distribution should be carefully 
defined to be the limit of a finite-interval distribution: 



where 



p{9) = lim pl{9), 

L— >oo 



pl(0) = ^; -L<e<L, 



in the case of a continuous distribution, or 
1 



PLi0) = 



= -L, -L + 1, 



,L, 



2L + 1 

in the case of an integer-valued distribution. The limit should 
be executed at the final process of the calculation of the 



marginal a posteriori distribution in order to avoid the in- 
determinate division which may be caused by the improper 
normalization coefficient. 

Appendix II 

Given AN and y^j^ 2 n)' 'he marginal a posteriori dis- 
tribution of /(l,2,...,n), or p(/(l,2,...,n) I A7V, y (1,2,. . . ,„) ), IS 

derived by integrations and the use of the Bayes theorem as 
follows: 

pi I {1,2,. ..,n) I A7V, y(i,2,. ..,„)) 
= J P{I{l,2,...,n), A-r(i^2,...,n) I AiV, y (1,2,. . .,„) ) 

X dAx(i^2, ...,«), 

P{I{l,2,...,n), Ar(i,2,...,n) I A7V, y(i,2,. ..,„)) 

J P( ^(1.2, ...,«), I{l,2,...,n), AT(i_2,...,n) I AiV, 

y(l,2,...,n)) dr(^i^2,...,n), 

P{r(l,2,...,n)\^N, y(i,2,. ..,„)) 

P{r{l,2,...,n), I{l,2,...,n), Ar(i^2,...,n) I A7V, 

y{l,2,...,n)) d,I(i^2,...,n)dAT (i,2,...,n), 
P{r(1^2,...,n), I{l,2,...,n), Ax(i^2,...,n) I AAT, y(i^2,...,n)) 

^p(7V°, r(^i^2,...,n), 1(1,2,. ..,n), ^'^ {l,2,...,n), 

T(l,2,...,n)^ V'(l,2,...,n) I AAT, y (1,2,. . . ,„) ) 

^ dT'('l,2,...,n)d'^P{l,2,...,n), 
.,n)7 -^(1,2, ...,n): ''"(1,2, ...,n) 7 "0(1,2,. ..,n) I 

AiV, 2/(1,2,. ..,„)) 

= p{N°, 'r(i_2,...,n), I{l,2,...,n), (1,2,. ..,n), '0(1,2,. .. ,n) : 

y(l,2,...,n)|AiV) /M 2/(1,2,. ..,n) I AiV), 
P{y (1,2,.. .,n)\^N) 

.,n), 1(1,2,. ..,n), '''(1,2, 

N« 

0(1,2, ...,n), 2/(1,2, ...,n) I AiV) 
X dr (^1^2,. ..,n)dl (1,2,. ..,n)dT(^i^2,...,n)d'4'(l,2,...,n)^ 
p{N°, r(i_2,...,n), -''(l,2,...,n), '''(1,2, , 

0(l,2,...,n), 2/(1,2, ...,„) I AiV) 
= -P( 2/(1,2,. ..,n) I AiV, N°, r(i^2,. ..,«), 1(1,2,. ..,n), 

'''(1,2,. ..,n), 0(1, 2,. ..,n)) 
Xp(iV°, r(i^2,...,n), -f(l,2,...,n), ^(1,2,. . . ,n) , 0(1,2,. .. ,n) ) , 
''■(1,2, ...,n) = Ax(i_2, ...,„) + (1, 1, . . . , 1)^ X 2,. ..,„), 

where a priori distribution, 

p{N^, r(i_2,. ..,„), -f(l,2,...,n), ''■(l,2,...,n), 0(1,2,. ..,«)), 

is used. 
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